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LAYERED IMAGE RENDERING 

FIELD OF THE INVENTION 

[0001] This invention pertains generally to computer graphics and 
image processing and, more particularly, to systems and methods for 
generating and compositing digital images, such as digital images 
composed of two or more image layers. 

BACKGROUND OF THE INVENTION 

[0002] Most computer graphics images are computed all at once, so 
that the rendering program takes care of all computations relating to the 
overlap and transparency of objects within the image. There are several 
applications, however, where objects within or layers of an image are 
rendered separately. In such a case, the compositing techniques used for 
combining the objects or layers into a single image are relied upon for the 
anti-aliased accumulation of a full image in which the transparency of 
foreground layer objects with respect to a background layer is properly 
represented. 

[0003] Standard digital RGB A images are defined at the pixel level by 
four values which define the red, green, and blue color components of the 
pixel, along with an alpha value which is a mixing factor indicating the 
amount of coverage or transparency of the pixel when composited with, 
e.g., overlaid on, a background image. For example, an alpha value for a 
pixel of zero may indicate no coverage or complete transparency, i.e., the 
pixel is clear, and thus would not in any way obscure a background layer 
pixel when composited therewith. An alpha value of one may mean full 

covBrage-or-a-completely_opaque_pixe^^ 

cover the corresponding pixel of a background layer composited with the 
image. Fractional alpha values correspond to partial coverage by or 
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transparency of the pixel when composited with a background layer 
image. 

[0004] When compositing computer graphics images of objects or 
image layers which overlap, standard equations are employed to combine 
the color elements of corresponding overlapping pixels in appropriate 
amounts defined by the alpha values of the pixels (e.g., the alpha value of 
the foreground layer pixel). This conventional color combining procedure 
works well in many cases. However, for certain foreground layer objects, 
particularly semi-transparent objects, to be overlaid on a colored 
background layer, such a compositing procedure employing a formulaic 
averaging of color contributions made by the foreground layer object and 
background layer image will not result in the desired composited image if 
the foreground layer is rendered in a conventional manner. For example, 
it may be desired to composite a semi-transparent foreground layer 
object, such as a graphical representation of a white snowflake, over a 
colored background layer. If the semi-transparent foreground layer object 
(the snowflake) is rendered in RGBA in a conventional manner, the 
compositing of the foreground layer object (the snowflake) and 
background layer in a conventional manner involving an averaging of the 
color contributions made by the foreground and background layer objects 
may result in an off-colored object (e.g., a gray snowflake) superimposed 
on a background layer, rather than the desired semi-transparent object 
(white snowflake) overlaid on the background layer. (A semi-transparent 
white snowflake over a colored background would be the resulting image 
if the entire image, foreground and background layers, were rendered at 
the same time.) Similar color artifacts can occur at the edges of opaque 
foreground layer image objects rendered in a conventional manner and 

overlaid-on-a background-layer imageajsing-conventionaLcompositing 

techniques. 
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[0005] What is desired, therefore, is a relatively simple method of 
creating a digital image that is composed of two or more image layers and 
in which the resulting composition will appear the same as if the scene 
layers were rendered as a single scene all at the same time. Preferably, 
such a method permits the creation of image layers which can be 
rendered separately and stored to be composited together at a later time, 
possibly with additional material added between the rendered image 
layers. 

SUMMARY OF THE INVENTION 

[0006] The present invention provides a method of creating a rendered 
scene that is composed of two or more image layers. Background and 
foreground image layers may be rendered separately, in accordance with 
the present invention, and saved to be composited at a later time. 
Additional material may be added between the rendered background and 
foreground image layers. The image resulting from composition of the 
image layers rendered in accordance with the present invention will 
appear as if all of the layers forming the image were rendered 
simultaneously as a single scene. 

[0007] All rendering in accordance with the present invention is done in 
RGBA. Background and foreground image layers to be composited at a 
later time to form a composited image are rendered separately. The 
elements of a background image layer may be rendered in a conventional 
manner in RGBA, and saved for later combination with a foreground 
image layer. (Alternatively, the background layer may be an RGB only 
image.) 

[0008] Creation of a foreground image layer in accordance with the 
~presenrinvention-begins™with~defining-a-scene™of™foreground~layeHmage~ 
elements or objects. The background of the foreground image layer thus 
defined is erased to full black. The elements of the foreground image 
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layer scene are then rendered in a conventional manner over the full black 
background to obtain RGB components for each pixel of the foreground 
image layer scene rendered over black. The background of the foreground 
image layer scene is then erased to full white. The elements of the 
foreground image layer scene are then rendered in a conventional manner 
over the white background to obtain RGB components for each pixel of 
the foreground image layer scene rendered over white. The RGB 
components for each pixel of the foreground image layer scene rendered 
over black are combined with the RGB components for each 
corresponding pixel of the foreground image layer scene rendered over 
white to form the rendered foreground layer image. This rendered 
foreground layer image may be composited with the saved background 
image layer, and/or other image layers, in a conventional manner, to form 
a composited scene which will appear the same as if the scene were 
rendered all at the same time. 

[0009] The step of combining the RGB components for each pixel of 
the foreground image layer scene rendered over black with the RGB 
components for each corresponding pixel of the foreground image layer 
scene rendered over white to form the foreground image layer may 
employ the following procedure in accordance with the present invention 
to obtain RGBA components for each pixel of the thus rendered 
foreground layer image. An alpha value for each pixel of the foreground 
layer image is determined as one plus the value of a color component of 
the corresponding pixel from the foreground image layer scene rendered 
over black minus the value of the same color component of the 
corresponding pixel from the foreground image layer scene rendered over 
white. For example, an alpha value for each pixel of the foreground layer 
image to be rendered may be determined as one plus the value of the red 
component of the corresponding pixel from the foreground image layer 
scene rendered over black minus the value of the red component of the 
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corresponding pixel of the foreground image layer scene rendered over 
white. If the resulting alpha value for the pixel of the foreground layer 
image scene to be rendered equals zero, all of the RGB color component 
values of the pixel are also set to zero. Otherwise, the RGB color 
component values of the pixel of the foreground layer image scene being 
rendered are set to the corresponding color component values of the 
corresponding pixel from the foreground image layer scene rendered over 
black divided by the alpha value for the pixel. The resulting RGB A 
foreground layer image is saved, and may be combined with the RGBA (or 
RGB) background layer previously rendered, using conventional 
compositing techniques, to create a composited image scene. 
[0010] The present invention may thus be employed in combination 
with conventional RGBA rendering techniques and programs to derive 
color components and alpha values for an RGBA foreground layer image 
to be composited with a background layer image. An RGBA foreground 
layer image rendered in accordance with the present invention may be 
composited with any background layer scene, and will retain the desired 
characteristics of the foreground layer image without blending of the 
colors of the foreground layer scene with the background layer image. 
Additional image layers, e.g., an image layer positioned between the 
foreground and background layers, may be composited to form a multi- 
layer scene. The desirable visual qualities of the foreground image layer 
rendered in accordance with the present invention are maintained in such 
a multi-layer composite. 

[0011] Further objects, features, and advantages of the invention will 
be apparent from the following detailed description taken in conjunction 
with the accompanying drawings. 
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BRIEF DESCRIPTION OF THE DRAWINGS 

[0012] Fig. 1 is a flowchart diagram of an exemplary method for 
creating a two-layer rendered scene image in accordance with the present 
invention. 

[0013] Fig. 2 is an exemplary background image layer scene to be 
composited with a foreground layer scene rendered in accordance with 
the present invention. 

[0014] Fig. 3 is an exemplary foreground image layer scene, with the 
background of the foreground image layer erased to full black, to be 
composited with the background image layer of Fig. 2. 
[0015] Fig. 4 is an exemplary image scene resulting from the 
composition of the background image layer of Fig. 2 with the foreground 
image layer of Fig 3, rendered in accordance with the present invention. 

DETAILED DESCRIPTION OF THE INVENTION 

[0016] An exemplary method 10 of creating a rendered multi-layer 
scene image in accordance with the present invention will be described in 
detail with referencte to the flowchart diagram of Fig. 1 . The exemplary 
method 10 to be described in detail herein features the compositing of 
two RGB image layers,\a background image layer and a foreground image 
layer. Each of the background and foreground image layers are rendered 
separately, and may be sa\ed and composited at a later time to form a 
composition which will appear the same as if it were rendered all at the 
same time. It should be understood that the present invention may be 
employed very effectively for tnte creation of rendered scenes that are 
composed of more than two RGB\rnage layers. For example, the present 
invention may be employed effectively to render the foreground image 
layer for a multi-layer virtual presentation incorporating three-dimensional 
graphics or animations with a live or pre-recorded two-dimensional video 



Atty. Dkt. No.: 067808:0118 

image positioned between foreground and background layers of the three- 
dimensional image. Such a Virtual Presentation System and Method with 
which the present invention may be employed is described in U.S. Patent 
Application No. 09^435,552, filed November 8, 1999, by Richard T. Daly, 
et al., which application is assigned to the assignee of the present 
application. Based oiS the detailed description presented below, a 
computer programmer^ ordinary skill in the art of computer graphics and 
image processing will ba able to implement a layered image rendering 
method in accordance w\h the present invention on a conventional 
computer system. * 

[0017] In accordance with the present invention, background and 
foreground image layers to be composited are rendered separately in 
RGBA. An RGBA background image layer is rendered at 12. The 
background image layer may be rendered in a conventional manner. The 
background image layer scene may include computer graphics generated 
images, and/or animated or still digitized photographic images, etc. The 
RGBA background image layer scene thus rendered is saved at 14, in a 
conventional manner, to be composited with a foreground image layer 
scene at a later time. Note that the background layer may be rendered as 
an RGB image with no alpha. If the background layer is rendered as an 
RGBA image, the alpha in the background layer may be used during 
compositing in a conventional manner, e.g., as a holdback mat. An 
exemplary background image layer scene 1 6, created in a conventional 
manner, is illustrated in Fig. 2. The exemplary background image layer 
scene 1 6 is a digital satellite image of North America. 
[0018] A foreground image layer is created at 1 8 by first defining the 
foreground image layer scene in a conventional manner. The elements, 
objectSy-or components appearing™in^the-foregroundJmage„layer_may_be — 
defined and created using any conventional computer graphics generation 
technique. An exemplary foreground image layer scene 20 is shown in 
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Fig. 3. The exemplary foreground image layer scene 20 is a graphical 
representation of the state borders of the United States. 
[0019] The background of the foreground image layer is erased to full 
black at step 22. In other words, the foreground image layer scene 
elements are defined over a full black background. (The exemplary 
foreground image layer elements 20 are shown in Fig. 3 with such a full 
black background.) 

[0020] The elements of the foreground image layer scene over the 
black background are rendered in a conventional manner at step 26 to 
obtain RGB components for each pixel of the foreground image layer 
scene rendered over black. We will refer to the resulting RGB image layer 
scene as FLB (foreground layer rendered over black). 
[0021] At step 28 the background of the foreground image layer is 
erased to full white. In other words, the elements of the foreground 
image layer scene are defined over a full white background. 
[0022] At step 30 the elements of the foreground image layer scene 
over the white background are rendered in a conventional manner to 
obtain RGB components for each pixel of the foreground image'layer 
scene rendered over white. We will refer to this rendered image as FLW 
(foreground layer rendered over white). (It should be noted that the 
sequence of steps 22, 26 and 28, 30 may be performed in any order, or 
simultaneously.) 

[0023] The RGB components for each pixel of the foreground image 
layer scene rendered over black (FLB) are then combined with the RGB 
components for each corresponding pixel of the foreground image layer 
scene rendered over white (FLW) at 32 to form the rendered foreground 
image layer (FL). A description of the preferred procedure for forming the 
foreground image layer (FL) from the foreground layer scene rendered 
over black (FLB) and the foreground layer scene rendered over white 
(FLW) follows. 
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[0024] For each corresponding pixel of the foreground image layer 
rendered over black (FLB) and white (FLW), a new RGBA foreground 
image layer (FL) pixel is created. The alpha value for each pixel of the 
new foreground image layer (FL) is determined as one plus the value of a 
color component (R, G, or B) of the corresponding pixel from the 
foreground image layer rendered over black (FLB) minus the value of the 
same color component of the corresponding pixel from the foreground 
image layer rendered over white (FLW). For example, an alpha value for a 
pixel of the new foreground layer (FL) may be determined as one plus the 
value of the red (R) component of the corresponding pixel from the 
foreground image layer rendered over black (FLB) minus the value of the 
red (R) component of the corresponding pixel from the foreground image 
layer rendered over white (FLW). 

[0025] If the resulting alpha value of the pixel is zero, the value of all of 
the RGB color components of the pixel are set to zero. Otherwise, the 
RGB color component values of the pixel of the new foreground layer (FL) 
are set to the corresponding color component values (R, G, and B) of the 
corresponding pixel from the foreground image layer scene rendered over 
black (FLB) divided by the alpha value for the pixel. This procedure is 
repeated for each pixel of the foreground layer (FL) to be rendered. The 
resulting foreground layer (FL) is saved at 34, to be combined with the 
background layer image, or other image layers. 

[0026] The procedure described above for rendering the foreground 
layer (FL) may be summarized by the following exemplary equations 
which are applied to generate each pixel of the foreground layer (FL): 
(FL alpha) = 1 + (FLB red component) - (FLW red component) 
If (FL alpha) = 0, then 

(FL red ) = 0 

(FL green) = 0 
(FL blue) = 0 
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Else 

(FL red) = (FLB red component)/(FL alpha) 

(FL green) = (FLB green component)/(FL alpha) 

(FL blue) = (FLB blue component)/(FL alpha) 



[0027] Thus, the present invention may be employed in combination 
with conventional RGB A rendering techniques and programs to derive 
color components and alpha values for an RGBA foreground layer image 
to be composited with a background layer image. The resulting rendered 
foreground layer (FL) may be combined with the saved background layer, 
using conventional compositing techniques, to form the composition 
image scene at 36. An exemplary composition 38 of the background 
image layer 1 6 of Fig. 2 with the foreground image layer 20 of Fig. 3, 
rendered in accordance with the present invention, is shown in Fig. 4. 
The resulting composition appears the same as if the entire image were 
rendered all at the same time, rather than resulting from the composition 
of separately rendered background and foreground image layers. A 
foreground image layer rendered in accordance with the present invention 
may be combined with any background layer image, or more than one 
background or intermediate layers, without adversely affecting the quality 
of the foreground layer as appearing in the composition image. 
[0028] It should be understood that the present invention is not limited 
to the exemplary applications and embodiments illustrated and described 
herein, but embraces such modified forms thereof as come within the 
scope of the following claims. In particular, it should be understood that 
a method in accordance with the present invention is not limited to the 
particular sequence of steps and equations described above. 
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